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PREFILTER DESIGN BY SPECTRAL FACTORIZATION 
BACKGROUND 

The present invention relates to communication systems and in particular to 
baseband signal processing in the receiving parts of base stations and mobile 
5 stations. The present invention provides a filter design method using spectral 
factorization that has a low complexity in implementation of the filter. 

The cellular telephone industry has made phenomenal strides in commercial 
operations in the United States as well as the rest of the world. Growth in major 
metropolitan areas has far exceeded expectations and is rapidly outstripping system 
10 capacity. If this trend continues, the effects of this industry's growth will soon 
reach even the smallest markets. Innovative solutions are required to meet these 
increasing capacity needs as well as maintain high quality service and avoid rising 
prices. 

In mobile communication, the transmitted signal is often subjected to a time 
1 5 smearing effect created by the time dispersive nature of the channel, i.e. , the air 
interface between a base station and a mobile station. The channel is estimated in 
the receiver part of a communication system, and used by the detector to aid in 
attempting to correctly deduce the information symbols that were transmitted 
thereto. A commonly used technique for deducing such received information 
2 0 symbols is Maximum Likelihood Sequence Estimation (MLSE) which, implemented 
using the Viterbi algorithm, is optimal for situations involving Additive White 
Gaussian Noise (AWGN). 

Although the MLSE can be implemented through a computationally efficient 
Viterbi scheme, the MLSE might nonetheless be very computationally complex if 
2 5 large symbol alphabets are employed or if the number of required taps in the 

channel estimate is large. In order to reduce the complexity of the Viterbi part of 
the equalizer, decision feedback can be used. In a Decision Feedback Sequence 
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Estimator (DFSE) equalizer, the estimated channel impulse response is split into 
two parts. The part which contains the first filter taps is referred to as the MLSE 
part and the remainder is referred to as the Decision Feedback Estimator (DFE) 
part. 

5 By implementing this hybrid MLSE/DFE structure, the computational 

complexity of the detector is reduced, but with some possible loss in performance 
compared to a pure MLSE detector implementation. If the energy of the channel is 
concentrated in the first channel taps, i.e. the MLSE part, then the performance loss 
is small. This can be accomplished by placing a prefilter in the receive signal 
10 processing chain. The purpose of the prefilter is to change the characteristic of the 
channel's impulse response such that the first channel taps are as large as possible. 
Furthermore, the prefilter should not change the magnitude response of the channel, 
or color the noise. 

Basically, several approaches have been proposed for constructing prefilters. 
15 A first approach to prefilter design employs Minimum Mean Square Error (MMSE) 
techniques. The MMSE prefilter design follows from the design of a pure Decision 
Feedback Equalizer (DFE). However, the complexity of the MMSE prefilter 
design is high. Furthermore, the MMSE prefilter may color the noise, making it 
undesirable as a prefilter for a hybrid MLSE/DFE equalizer since the Viterbi 
2 0 MLSE part is not optimal for signals impacted by colored noise. 

Second, allpass filters have been used as prefilters so that the new channel 
estimate becomes a minimum phase filter. These kinds of prefilters have been 
designed using a root search algorithm. The root search algorithm consists of an 
exhaustive sequential search that identifies each root one at a time. For example, in 
2 5 a 7th order channel estimate, seven roots have to be estimated. The first root has to 
be found using an iterative search method (e.g., Newton Raphson, such as 
described in Chapter 7 of Soderstrom and Stoica, System Identification. Prentice 
Hall 1989). Next, the root search method performs long division of the channel 
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estimate with the first root. Thus, a 6th order channel remains after the division. 
A new iterative search, but now based on this 6th order channel, results in a second 
root. The method continues the process of iterative search and long division until 
the channel is reduced to a 4th order system. Then, the remaining four roots can be 
5 found by closed form equations. After all the seven roots of the channel estimate 
are calculated, all roots which are outside the unit circle are mirrored inside the unit 
circle, which results in a set of seven roots inside the unit circle. Based on this set 
of roots inside the unit circle, a minimum phase channel is constructed. Also, a 
prefilter is constructed based on this set of roots. However, the indirect nature of 
10 the root search algorithm is computationally complex. 

A third approach is described in an article (Gerstacker et al., "An Efficient 
Method for Prefilter Computation for Reduced-State Equalization", Proceedings of 
the PIMRC 2000 conference, London, UK, September 18-21, 2000) and related 
European Patent Application EP 1032170 Al. The third approach consists of 
15 calculating a two stage FIR prefilter. The first FIR filter is a matched filter, i.e., 
the estimated channel conjugated and reversed. The second FIR filter consists of (a 
FIR approximation of) the inverse of the minimum phase version of the channel 
filter, which is also conjugated and reversed. The latter FIR filter is calculated 
from the solution of a set of Yule-Walker equations. This approach is more 
2 0 sensitive to truncation in the FIR approximations due to the two stage filtering, 
compared to the method described in the present document. Additionally, in 
section three of the article the authors detail other approaches that have lead to 
unsatisfactory solutions due to computational complexity. Among these other 
approaches is a spectral factorization technique which uses the cepstrum of the 
2 5 channel autocorrelation sequence. The authors note that the computational 
complexity is quite high and requires extensive logarithmic and exponential 
operations. 
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A spectral factorization technique which does not use the cepstrum is 
described in an article by Jezek, J., and Kucera, V., "Efficient Algorithm for 
Matrix Spectral Factorization", Automatica, Vol. 21, No. 6, pp. 663-669, 1985. 
However, this method is designed for real valued filter coefficients and does not 
5 allow for complex valued filter coefficients. 

Thus, it would be desirable to generate new prefilter designs which are low 
in complexity, which do not color the noise, and which are implemented in a single 
stage while at the same time concentrating signal energy in the MLSE taps of the 
detector . 

L0 SUMMARY 

It should be emphasized that the terms "comprises" and "comprising", when 
used in this specification, are taker, to specify the presence of stated features, 
integers, steps or components; but the use of these terms does not preclude the 
presence or addition of one or more other features, integers, steps, components or 

15 groups thereof. 

The present invention describes a spectral factorization method that is a low 
complexity alternative way of constructing prefilters, requiring a fraction of the 
computational complexity compared to conventional prefilter designs. Furthermore, 
the prefilter design of the present invention does not color the signal noise and 
2 o operates as an allpass filter and can be realized in a single stage. Still further, the 
required program memory is significantly reduced by the spectral factorization 
method that does not involve using the cepstrum of the channel autocorrelation 
sequence or require extensive logarithmic and exponential operations. The spectral 
factorization method also is less sensitive to quantization noise in signals and 
25 channel estimates. Additionally, the invention describes how to construct prefilters 
to be used for equalization in both the forward and backward direction. 
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The present invention overcomes the computational complexity and 
inefficiency of the root search algorithm by directly calculating a minimum phase 
version, G(z), of a channel estimate, H(z) and calculating the prefilter form the 
minimum phase. One exemplary method provides for determining filter coefficients 
in a communication system by estimating a channel over which signals have been 
transmitted and determining the filter coefficients for a filter from the channel 
estimator using a spectral factorization method. Another exemplary method begins 
by initializing coefficients of the minimum phase filter, g, to coefficients of a 
spectrum of a channel estimate, s, wherein each coefficient numbers from 0 to L. 
An outer iterative loop starts by calculating a division factor, y 0 , from a complex 
conjugate of a first coefficient of the minimum phase filter G(z). The outer iterative 
loop starts by calculating a next L"< coefficient, x L , of the minimum phase filter by 
multiplying the L !h term uf the spectrum with the division factor. A value m 
(starting at m=L-l) of an inner iterative counter is used and the next m ll > 
15 coefficient, x m , is calculated by an equation as follows: 

' L ^ 



10 



20 



X = 
m 



i=m-M 



^0 



25 



wherein g is the complex conjugate of g. The value m is checked to see if m is 
equal to zero. If m is not zero, the value m is decreased by one and the process 
loops back to the start of the inner iterative loop. If m is equal to zero, the 
coefficients of the minimum phase filter, g, are set equal to the next coefficients of 
the minimum phase filter, x, for each coefficient from 0 to L. An outer iterative 

loop criteria is then checked. If the outer iterative loop criteria is met the process 

ends. If the outer iterative loop criteria is not met the process loops back to the 

start of the outer iterative loop. 

The above features and advantages of the present invention will be more 

apparent and additional features and advantages of the present invention will be 
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appreciated from the following detailed description of the invention made with 
reference to the drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention will now be described with reference to the following 

5 figures, in which: 

Fig. 1 shows a block diagram of the parts of the baseband signal processing 

with an equalizer of decision feedback type; 

Fig. 2 illustrates the prefiltering of the right and left data parts for the GSM 

system; 

10 Fig. 3 is a flow diagram of the iterations for spectral factorization according 

to the present invention; 

Fig. 4 is a graph of the roots of a channel in a GSM system and the roots of 
the resulting from the spectral factorization method; 

Fig. 5 is a graph of the spectra of a channel in a GSM system and spectra of 
15 prefilters from various design methods; 

Fig. 6 is a graph of the roots of another channel in a GSM system; 

Fig. 7 is a graph of the spectra of a channel in a GSM system for various 
filters when to few prefilter coefficient are used; and 

Fig. 8 is a block diagram of a conventional radio communication system. 

20 DETAILED DESCRIPTION 

In the following description, for purposes of explanation and not limitation, 
specific details are set forth, such as particular circuits, circuit components, 
techniques, etc. in order to provide a thorough understanding of the present 
invention. However, it will be apparent to one skilled in the art that the present 
2 5 invention may be practiced in other embodiments that depart from these specific 
details. In other instances, detailed descriptions of well-known methods, devices, 
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15 



20 



25 



and circuits are omitted so as not to obscure the description of the present 
invention. 

The exemplary radio communication systems discussed herein are based 
upon the time division multiple access ("TDMA") protocol, in which 
communication between the base station and the mobile terminals is performed over 
a number of time slots. However, those skilled in the art will appreciate that the 
concepts disclosed herein find use in other protocols, including, but not limited to, 
frequency division multiple access ("FDMA"), code division multiple access 
("CDMA"), or some hybrid of any of the above protocols. Likewise, some of the 
exemplary embodiments provide illustrative examples relating to the GSM or IS- 136 
type of systems; however, the techniques described herein are equally applicable to 
radio communication systems operating in accordance with any specification. 

Prior to discussing exemplary embodiments according to the present 
invention, Fig. 8 will now be described which illustrates a general radio 
communication system 100 in which the present invention can be implemented. 
The radio communication system 100 includes a plurality of radio base stations 
170a-n connected to a plurality of corresponding antennas 130a-n. The radio base 
stations 170a-n in conjunction with the antennas 130a-n communicate with a 
plurality of mobile terminals (e.g. terminals 120a, 120b, and 120m) within a 
plurality of cells llOa-n. Communication from a base station to a mobile terminal 
is referred to as the downlink, whereas communication from a mobile terminal to 
the base station is referred to as the uplink. 

The base stations are connected to a mobile telephone switching office 
("MSC") 150. Among other tasks, the MSC coordinates the activities of the base 
station, such as during the handoff of a mobile terminal from one cell to another. 
The MSC, in turn, can be connected to a public switched telephone network 160, 
which services various communication devices 180a, 180b, and 180c. Both the 
mobile terminals 120a, 120b, and 120m, and the base stations 170a-n can 
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incorporate prefilter design structures and techniques according to the present 
invention. 

A detailed description of prefilter design and prefiltering according to the 
present invention is given below. The description contains examples using the burst 
5 format defined for the GSM standard, but as will be appreciated by those skilled in 
the art the invention is not limited to this particular standard and can be 
implemented in communication systems designed in accordance with other 
standards. 

An exemplary block diagram of a receiver chain, and a time dispersive 
10 channel over which a signal can be transmitted, is illustrated as Fig. 1. A 

synchronization and channel estimator unit 1 1 produces a channel estimate H(z) of 
the channel 10. The received signal y(n) is prefiltered with the prefilter 12, using 
filter coefficients established by the transfer function F rwii (z). The output of prefilter 
12 results in the signal y fwd (n). The minimum phase (MP) filter G rwd (z), is 
15 generated in prefilter design block 16. The minimum phase filter, G fwd (z), is used 
by the equalizer 14 and has its energy concentrated to the first filter taps. 

The prefilter design results in two versions of each filter, one to be used in 
the forward direction of the burst (with index fwd ), and one for the backward 
direction (with index bwd), as shown in Fig. 2. Therein, data portions defined for 
2 0 frames in the GSM system are illustrated and consist of a ramp block 200 and tail 
block 202 proceeding the left data burst 204 and training block 206 followed by the 
right data burst 208, tail 210 and ramp 212 blocks. The filter for the forward 
direction is used for equalization of the right half of the data burst, from training 
206 to ramp 212, and the backward version for equalization of the left half of the 
2 5 burst, from training 206 to ramp 200. 

One property of a minimum phase filter G(z) is that all the roots of the filter 
polynomial are located inside the unit circle. This property will be used in the 
analysis below. This property of a minimum phase filter will also result in 
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prefilters which are allpass filters, i.e. in prefilters having a substantially flat 
frequency response. These prefilters are of an Infinite Impulse Response (IIR) type, 
with a rational transfer function given as: 

B ( Z ) M\ 

F(z) - — — (1) 
A(z) 

where each root z, of the numerator B(z,) = 0 is located outside the unit circle. The 
5 roots of the denominator are 1/z,*, i.e. the roots of the numerator mirrored in the 
unit circle. These IIR filters can be approximated by Finite Impulse Response 
(FIR) filters. In general, IIR filters are avoided in fixed point DSP implementations 
since rounding errors might result in unstable filters. Furthermore, as will be 
clarified in the section below regarding prefilter design, this approximation is useful 
10 in the design of prefilters based on the spectral factorization method. 

An example of a spectral factorization algorithm according to an exemplary 
embodiment of the present invention is given below. This iterative algorithm is 
designed to deal with complex valued filter coefficients. Furthermore, the 
algorithm leads to a simple system of linear equations to be solved for each 
1 5 iteration, thereby facilitating implementation of the prefilter design. 

For example, denote the Z-transform of the channel estimate by: 

H(z) = hz L + h iZ L -' + ... + h L . (2) 



This channel estimate is in general not minimum phase. The purpose of the spectral 
factorization method is to find a minimum phase channel G(z) such that: 

|H(e jw )| 2 = |G(e ju )| 2 , 0<o><27t (3) 



where: 
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= H(e ju ) 



(4) 



The magnitude response is thus identical for H(z) and G(z) but the phase 
responses are different. The spectrum of the channel estimate is: 



S(z) = H(z)H '(z"') 



s .z L +s_, Z b ^ ...+ s 0 + ... + s L .,z- Li| + s L z 



(5) 



-L-l 



Since H(z) has complex valued coefficients in general, this spectrum has 
complex conjugated symmetric coefficients, i.e. s. s = s*. 
5 The minimum phase version of the channel estimate is modeled as: 



G(z) = g_z L + g.z 



L-l 



+ ...+ 



(6) 



The coefficients of the spectrum S(z) can be expressed through the coefficients of 
G(z) as: 

L -m 

s e *p for 0 £ m < L, 

1=0 



(7) 



or, equivalently, in a matrix formulation: 

, 0 &o - 6t-i 
s = = 

0 

0 0 g n 



= T , (g)g. 



(8) 
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The criteria which is considered is: 

V =||s-T'(g^^)gi (9) 

where g (k) denotes the estimated coefficients at iteration k and g^ ]) denotes the 
previously estimated coefficients. The algorithm for finding a minimum phase 
channel is based on finding values of the coefficients of g (k) such that the criteria V 
in equation (9) is minimized. The derivative of V with respect to g (k> is T*(g (W »). 
A Newton Raphson search, is thus: 

g ( k+ n = g w + T'"(gW)(s-T'(g (l) )g lk) ) = (tW's 



(10) 



where g (k l) has been replaced by g lk) . Repeat equation (10) until: 

v =|s-T-(g (k -V k) lh (U) 



for some value of e. The stop limit of the criteria, e, is a design variable the value 
of which determines the requested accuracy of the minimum phase channel. The 
10 vector g is initiated, for k = 0 as: 

g» = s. C12) 

Those skilled in the art will appreciate that other values, e.g., within the unit 
circle, can be used to initiate vector g. If the values specified for initialization of 
the vector g are outside the unit circle, then a maximum phase filter will be 
generated, from which the minimum phase can be obtained. 
15 Experiments have indicated that equation (10) should be iterated 

approximately 10 times for a GSM channel to provide sufficient accuracy. Those 
skilled in the art will appreciate that the number of iterations can be greater or less 
than 10. Spectral factorization methods according to the present invention permit 
determining complex valued filter coefficient using a relatively few number of 
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iterations, e.g., less than about 30. The system of equations in equation (10) can be 
solved in approximately V2L 2 multiplications. It may be desirable to implement 
algorithms according to the present invention using a fixed number of iterations 
instead of permitting a variable number of iterations halted by a limit on the criteria 
5 in order to be able to exactly predict the computational requirements for the 
algorithm. 



Solving triangular an d Toeplitz systems 

The algorithm for finding the spectral factor G(z) is 

Let x = g <k+1) , g = g (k> and solve the following system of equations for x: 



(13) 



L-l 

S. 



^1 

0 g n 



Bl-i 



0 g n 



L-l 
X. 



(14) 



10 



Starting with the last row of equation (14) yields the solution to x L as: 

\ - (15) 



The value of x L ., can now be computed as: 



06) 
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The system is solved by iterating upwards in the system of equations (14). Note 
that the calculations above can be performed using only additions, multiplications 
and one division. The division is performed by defining: 

Y 0 " (17) 



10 



15 



and inserting Yo instead of 1 /go' above. The absolute value of g 0 * is not close to 
zero since it is the first coefficient of a minimum phase filter. The algorithm is 
illustrated below and in Fig. 3. 

Referring to Fig. 3, block 300 initializes g; = s, for m=0,...,L. Next in 
block 302, Yo, *l and counter m are all initialized. Blocks 304, 306 and 310 form 
an inner iterative loop that solves equation (14) by sequentially solving for x m at 
each iteration. An outer iterative loop is formed by blocks 302, 320, 322 and 324. 
In decision block 306, when m has counted down to zero, the process proceeds to 
block 320 and sets g, = x, <g»=g° +, \ see equations (13) and (14)). The process 
then continues to block 322 to check if the outer counter k is equal to K. Upon a 
negative decision in block 322, the process increments counter k in block 324 and 
feeds the updated values for g, into block 302. The process then continues until the 
k = K in decision block 322. 

The spectral factorization method, that corresponds to Fig. 3, is presented 
below in pseudo code to further illustrate the present invention. 



20 



for m=0 to L 

L -m 



|a0 /* Calculate spectra */ 

gm - S(n /* Copy result from s to g */ 



end 

for k = 0toK-l 
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(14)*/ 



Yo = 1 / go* 
x L = s L Yo 

for m = L-l down to 0 /* Iteration upwards in 



/*The last row of (14)*/ 



x = 

m 



U 

S m ~£ X i g i-n 
ism- 1 ! 



end 

for m=0 to L /* Copy result from x to g */ 

gm = X n» 

end 



end. 



10 



Prefilter Design 



Fnrwarrl Prefilter Calculation 

A forward prefilter calculation is described wherein the result from the 
iterative solution above (coefficients g, ) is denoted as G min (z) which is the minimum 
phase version of H(z). This minimum phase filter is used for the DFSE of the right 
15 data burst, i.e. in the forward direction: 



G (z) = G . (z). 



The channel estimate is modeled by: 



(18) 



H(z) = D[ (*-*,) II C z ~ z ,) (19) 
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where the roots inside and outside the unit circle are denoted by: 

z GO if |z.| 2 ^l and z.eQ. if |z.| 2 <l respectively. (20) 

i ^out i i in i 

The estimated minimum phase version of H(z) has the following structure: 




(21) 



where the tilde above in the z. roots indicates the error in the calculation of G min by 
the spectral factorization. 

The prefilter for the forward direction can be expressed as: 

F (z) . Ml . n ^ n ^ ™ 



10 



15 



The second factor on the right hand side of equation (22) can be neglected when the 
estimated G min has roots close to the roots, inside the unit circle, of H(z). Note that 
F fwd (z) is an allpass filter when the second factor is small, which implies a flat 
frequency spectrum. The IIR filter in equation (22) is not used directly in causal 
filtering since the denominator contains roots outside the unit circle. Moreover, 
this filter is not readily implemented in noncausal filtering either due to the 
presence of roots inside the unit circle. This IIR prefilter can, instead, be 
approximated by a FIR filter by using a long division to approximate the 
denominator of the polynomial. Of course, since this is an approximation it will 
introduce error. The error in this long division, or approximation of the 
denominator of the polynomial, typically should decrease as the number of FIR taps 
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increases. However, H(z) contains roots both inside and outside the unit circle, 
which should not be problematic for the long division since G fwd and H(z) contain 
some common roots inside the unit circle which should cancel each other. 
However, due to possible estimation errors in the spectral factorization, these roots 
5 may not cancel completely resulting in a situation where the error in the 

approximation of the FIR filter actually increases when the number of FIR taps is 
increased. 

To provide for a better approximation, the present invention provides an 
alternative technique wherein one first computes: 

F i (z> = rwzl " f o z " * f i z "'' * "' * f " ' (23) 

rwd 1 - ' 

1 o where M is the number of taps in the FIR approximation of the IIR filter. 

Simulations have indicated that M should be some 2-3 times larger than L. The 
forward filter F fwd is obtained by reversing the order and conjugation of the 
coefficients of F^z) or, equivalently, 

FJ*)=V M F>-') (24) 

where a is a constant which scales the forward filter without changing the filter 
15 performance: 

The long division above, in equation (24), is an approximation of an IIR 
filter with a FIR filter. The error in this approximation will then decrease when the 
number of coefficients M of the FIR filter increase. It is thus desirable that the 
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prefilter F rwd is computed as outlined in equations (23) and (24) instead of equation 
(22). 



Backward Prefilter Calculation 

In calculating a backward prefilter, let: 



H(z) = z L H(z 



(26) 



where the overbar thus denotes a reversing of the coefficients. The overbar also 
indicates a filter to be used on the reversed signal. This reversed polynomial has 
roots at: 



k(z) = n 

!.«Q, 



f 0 

z- — 


n 


f 0 

z- — 


z. 

V x ) 




z. 

\ v 



(27) 



The backward minimum phase filter is now a filter which is a minimum 
phase version of H(z) . This backward filter can be expressed as: 



5 bwd (z) = n n 



z- — 
z. 

V '/ 



(28) 



10 By comparing equations(18),(21) and (28) it follows that the backward minimum 
phase filter is computed as: 

G Xz) = G '. (z) (29) 



i.e. the backward filter is achieved by conjugating the coefficients of the minimum 
phase version of the estimated channel filter. The prefilter to be used in the 
backward direction is: 
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f 5bwd(z) n ^ n ^ 

bwdCZ) " H(z) " h \i ^Q eul *-lA f 



(30) 



The second factor on the right hand side of equation (30) can be neglected when the 
estimated G has roots close to the roots, inside the unit circle, of H(z) . Note 

bwd 

that F bwd (z) is an allpass filter, when the second factor is small, which implies a flat 
spectrum. This prefilter can be approximated by a FIR filter with M taps by using 
long division. But, as described above, since H(z) contains roots both inside and 
outside the unit circle, this long division is not numerically efficient. To improve 
on the above long division method, first compute: 



F(z)= J?W_ B f Z M +fz M-, + ... +f (31) 



The backward filter F bttfd is obtained by reversing the order and conjugation 
of the coefficients of F b (z), or, equivalently, 

■ v Mf >") (32) 



10 where: 



(33) 
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All pass Properties 

The whiteness of prefilters designed according to the spectral factorization 
methods described herein depends on the number of iterations of equation (10) (and 
the error in the approximation of the IIR prefilter with an FIR filter) which are 
preferred. This can be seen in equation (22) and (30) where the accuracy of the 
spectral factorization is modeled by the closeness of \ to z,. 

In the prefilter design by the root search method, the roots of the channel 

filter t for i = 1 L, are explicitly calculated. The prefilter is calculated 

directly from these estimates as: 

z-l/z.* 



20 



f = n — ^ 

fwd.iool " z- z 



25 



The prefilter is then, by construction, an allpass filter, as long as the FIR 
approximation of equation (34) is not truncated too early. The minimum phase 
filter is then constructed by a convolution of this prefilter by the original channel 
filter. To further appreciate various qualities of filter design according to the 
present invention, consider the following simulation results. 

Referring to Fig. 4, the roots of a channel filter and corresponding roots 
mirrored in the unit circle for a Hilly Terrain channel, as defined by GSM 
specification 5.05, are shown. Three sets of symbols illustrate these roots. The 
first set of symbols are the roots of channel filter (+), such as 420. The second set 
of symbols are the roots mirrored inside the unit circle (x), such as 410. The third 
set of symbols are the roots of the minimum phase filter found by the spectral 
factorization, as described above, are displayed as circles (o), such as 430. The 
roots found by the spectral factorization method were identified by iterating 
equation (10) ten times. Fig. 4 illustrates a minimum phase filter and its 
relationship to a channel filter that is not minimum phase. The spectral 
factorization method yields, as is shown by a comparison of the roots, essentially 
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the same minimum phase filter. The spectral factorization method of the present 
invention achieves this result using significantly less processing power than 
conventional methods. All roots of the channel filter are not shown in Fig. 4, 
because one root lies significantly outside the unit circle. Table 1 lists the values as 
plotted in Fig. 4, and all the roots for the channel filter. 



10 



Roots of the channel 
filter 


Roots mirrored in the 
unit circle 


Roots of the minimum 
phase filter 


0.5489 + 0.5581i 


0.8958 + 0.9107i 


0.5489 +0.558H 


0.4720 -0.6543i 


0.7384- 1.0095i 


0.4720 - 0.6453i 


-0.7338 + 0.6072i 


-0.8089 + 0.6694i 


-0.7338 + 0.6072i 


-0.8871 -0.5104i 


-0.8469 - 0.4873i 


-0.8469 - 0.4873i 


-3.2978 + 0.1205i 


-0.3028 + O.Ollli 


-0.3028 4- O.Ollli 



The graph of Fig. 5 shows the spectrum of the channel filter 502, which 
15 corresponds to the channel filter of Fig. 4. The spectra of the prefilters from the 
root search algorithm 504 and from the spectral factorization 506 are also shown. 
It can be seen that the spectrum of the root search 504 is flatter than the spectrum of 
the spectral factorization pref.lter 506. The minimum phase filter estimate becomes 
more accurate as the number of iterations of the spectral factorization is increased. 

2 o An increase in the number of iterations also results in a flatter ("whiter") spectrum 
of the prefilter, i.e. a more ideal altpass response. However, an increase in the 
number of iterations also increases processing time. Therefore, the number of 
iterations performed should provide an acceptable result, while minimizing 
processing time, especially for real time systems. 

2 5 Referring to Fig. 6, the roots of a channel filter and corresponding roots 

mirrored in the unit circle for a different channel simulation are shown. Three sets 
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of symbols illustrate these roots. The first set of symbols are the roots of channel 
filter ( + )■ such as 610. The second set of symbols are the roots mirrored inside the 
unit circle (x), such as 620. The third set of symbols are the roots of the minimum 
phase filter found by the spectral factorization, as described above, are displayed as 
circles (o), such as 630. The roots found by the spectral factorization method were 
identified by iterating equation (10) ten times. Fig. 6 illustrates a minimum phase 
filter and its relationship to a channel filter that is not minimum phase for another 
channel simulation. The spectral factorization method yields, as is shown by a 
comparison of the roots, essentially the same minimum phase filter. All roots of 
the channel filter are not shown in Fig. 6, because two roots lie significantly outside 
the unit circle. Table 2 lists the values as plotted in Fig. 6, and all the roots for the 
channel filter. 



Roots of the channel 
filter 


Roots mirrored in the 
unit circle 


. — . 1 

Roots of the minimum 

phase filter 


0.9970 + 0.2052i 


0.9622 + 0.1981i 


0.9622 + 0.1981i 


-0.3429 + 1 .046li 


-0.2830 + 0.863H 


-0.2830 + 0.8631i 


0.1142 - 1.15861 


0.0843 - 0.8548i 


0.0843 - 0.8548i 


-1.2868 -0.1622i 


-0.7650 - 0.0964i 


-0.7650 - 0.0964i 


-2.6979 - 0.6838i 


-0.3483 -0.0883i 


-0.3483 - 0.0883i 



20 



25 



Referring to Fig. 7, another trade-off in the construction of minimum phase 
prefilters is illustrated. This problem arises when the roots are located close to the 
unit circle. The spectra of the channel filter 702 is plotted with the spectra of both 
the root search algorithm 704 and the spectral factorization 706 prefilter designs. 
The spectrum of the channel filter 702 corresponds to the channel filter of Fig. 6. 
The spectra of both the root search algorithm 704 and the spectral factorization 706 
prefilter designs have large dips, 710 and 708, respectively. The dips 708 and 710 
become smaller as the number of taps in the prefilter increases. In this example, it 



WO 02/33923 




PCT/SE01/02262 



-22- 



can be seen that the long division in equation (23) (and equation (31)) is truncated 
too early. The "whiteness" of the prefilter can thus be improved by increasing the 
number of taps of the prefilter. The prefilters of both Fig. 5 and Fig. 7 have the 
same number of taps. However, due to the different channel filters, the number of 
5 taps in the latter case was not sufficient. Although the root search algorithm is much 
more computationally complex and exhaustive, as shown in Fig. 7, the resulting 
prefilter is not better than that achieved by the spectral factorization method of the 
current invention. 

The invention provides a design method for calculating minimum phase 

10 prefilters. The method is based on using a spectral factorization approach. The 
prefilter is constructed from a minimum phase version of an estimated channel 
impulse response. A system of linear equations is solved for each of the iterations 
of the spectral factorization. The invention is, unlike most other spectral, 
factorization methods, designed to deal with complex valued filter coefficients. 

15 From an implementation point of view, the present invention is appealing because it 
requires only multiplications, additions and one division for each of the iterations. 
The proposed method of calculating the prefilter becomes very important as the 
number of prefilter coefficients are increased. Additionally, the invention describes 
how to construct prefilters to be used for equalization in both the forward and 

2 0 backward direction. The reduced complexity of the current prefilter design allows 
for the prefilter to be implemented on processors with reduced computing power. 

The minimum phase filter improves the performance of an equalizer using 
decision feedback. The invention makes the implementation of the minimum phase 
filter design possible in fixed point Digital Signal Processors (DSPs). 

2 5 The foregoing examples have been described in the context of prefilters used 

in hybrid MLSE/DFE systems. However, those skilled in the art will appreciate 
that the present invention can be implemented in systems using any type of 
equalizers, e.g., pure MLSE equalizers. 
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Therefore, the above-described embodiments should be regarded as 
illustrative rather than restrictive, and it should be appreciated that variations may 
be made in those embodiments by workers skilled in the art without departing from 
the scope of the present invention as defined by the following claims. 
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CLAIMS 
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What is claimed is: 

1. A method for determining filter coefficients in a communication system 
comprising the steps of: 

estimating a channel over which signals have been transmitted; and 
determining said filter coefficients for a filter from said channel estimates 
using a spectral factorization algorithm. 

2. The method of claim 1, wherein said spectral factorization algorithm 
generates at least one complex valued filter coefficient. 

3. The method of claim 2, wherein said step of determining said filter 
coefficients using spectral factorization algorithm comprises the step of: 

interatively solving a system of linear equations. 

4. The method of claim 3, wherein said step of determining further comprises 
the step of: 

performing less than 30 iterations to determine said filter coefficients. 

5. The method of claim 3, wherein solving the system of linear equations 

comprises the steps of: 

(a) determining spectrum coefficients of a spectrum of said channel 

estimate; 

(b) initializing current filter coefficients; 



of the current 



(c) 



creating a complex conjugate matrix containing complex conjugates 
filter coefficients; 
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(d) multiplying the inverse of said complex conjugate matrix by a vector 
containing said spectrum coefficients to obtain next filter coefficients; 

(e) setting said current filter coefficients as equal to said next filter 
coefficients; and 

5 (f) checking a loop criteria, wherein if said loop criteria is met ending, 

else looping to step (c). 

6. The method of claim 5, wherein said step of initializing comprises the step 
of: 

setting said current filter coefficients equal to said spectrum coefficients. 

10 7. The method of claim 5, wherein said loop criteria is met by looping a fixed 
number of times. 

8. The method of claim 5, wherein said loop criteria is met by: 

V -l.-T-te^V^e 

9. The method of claim 1, wherein said filter coefficients are a minimum phase 
version of said channel estimate. 

15 10. The method of claim 1, wherein said filter coefficients are a maximum 
phase version of said channel estimate. 

11. The method of claim 1 , further comprising the steps of: 

determining at least one of forward and backward prefilters using said filter 

coefficients. 
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12. The method of claim 1 1 , further comprising the steps of: 
approximating at least one of forward and backward prefilters as Finite 

Impulse Response (FIR) filters. 

13. The method of claim 12, wherein the step of approximating at least one of 
5 the forward and backward prefilters further comprises the steps of: 

determining intermediate coefficients for at least one prefilter; 
reversing the order of said intermediate coefficients for at least one prefilter; 
conjugating said intermediate coefficients for at least one prefilter; and 
scaling said reversed and conjugated intermediate coefficients to determine 
10 at least one prefilter. 

14. The method of claim 1 wherein determining the forward prefilter further 

comprises the steps of: 

calculating the forward prefilter according to: 

and 



15 



15. The method of claim 11, wherein determining the backward prefilter further 

comprises the steps of: 

calculating the backward prefilter according to: 
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16. The method of claim 1, further comprising the steps of: 
receiving a channel input; 

determining a channel estimate using said channel input 
5 supplying said channel estimate to said spectral factorization algorithm; and 

supplying said filter coefficients to an equalizer. 

17. The method of claim 1, further comprising: 

determining both forward and backward prefilters with said filter 

coefficients. 

10 18. The method of claim 1, wherein said spectral factorization algorithm does 
not use the cepstrum of the channel autocorrelation sequence. 

19. The method of claim 1, wherein said filter coefficients are determined in a 
single stage FIR approximation. 

20. A transceiver in a communication system comprising: 

15 a filter; 

a channel estimator that est.mates a channel over which signals have been 

transmitted; and 

a filter design unit that performs a spectral factorization algorithm that 
determines filter coefficients from said channel estimates, wherein said filter 
2 o coefficients are used to calculate said filter. 
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21 . The transceiver of claim 20, wherein said transceiver is base station 
transceiver. 

22. The transceiver of claim 20, wherein said transceiver is mobile terminal 
transceiver. 

5 23. The transceiver of claim 20, wherein said spectral factorization algorittun 
generates at least one complex valued filter coefficient. 

24. The transceiver of claim 20, wherein said filter coefficients are a minimum 
phase version of said channel estimate. 

25. The transceiver of claim 20, wherein said filter coefficients are a maximum 
10 phase version of said channel estimate. 

26. The transceiver of claim 20, wherein a prefilter has both forward and 
backward prefilters that are designed using said filter coefficients. 

27. The transceiver of claim 20, further comprising: 

an equalizer for compensating said radio signals for effects of said channel. 

15 28. The transceiver of claim 27, wherein said equalizer is a Decision Feedback 
Sequence Estimator (DFSE) equalizer. 

29. The transceiver of claim 28, wherein said filter is a prefilter that focuses 
energy of said radio signals in taps associated with an MLSE portion of said DFSE 
equalizer. 
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30. The method of claim 20, wherein said spectral factorization algorithm does 
not use the cepstrum of the channel autocorrelation sequence. 

31. The method of claim 20, wherein said filter coefficients are determined in a 
single stage FIR approximation. 

5 32. A method of calculating a minimum phase version, G(z), of a channel 

estimate, H(z), for implementing a prefilter using spectral factorization, comprising 
the steps of: 

(a) initializing coefficients of the minimum phase filter, g, to coefficients of 
a spectrum of a channel estimate, s, wherein each said coefficients number from 0 

10 to L; 

(b) calculating a division factor, Yo > from a complex conjugate of a first 
coefficient of said minimum phase filter; 

(c) calculating a next L" 1 coefficient, x L , of the minimum phase filter by 
multiplying the L lh of said spectrum with said division factor; 

(d) establishing a value m of an inner iterative counter as one less than L; 

(e) calculating the next m* coefficient, x m , by an equation as follows: 



15 



20 



x = 



\ m i=mM 



\ 



wherein g' is the complex conjugate of g; 

(f) checking the value m, wherein if the value m is equal to zero continue to 
step (g), else decrement the value m by one and loop to step (e); 

(g) equating the coefficients of the minimum phase filter, g, to the next 
coefficients of the minimum phase filter, x, for each coefficient from 0 to L; and 
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(h) checking an outer iterative loop criteria, wherein if said outer iterative 
loop criteria is met ending, else loop to step (b). 
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